Skip to content

Conversation

@victorges
Copy link
Member

No description provided.

Refactor pipeline discovery to use Python entry points, enabling external
pipelines to live in separate repositories and be installed as dependencies.

Key changes:
- Created pyproject.toml for ai-runner-base package with entry points
- Refactored loader.py to discover pipelines via entry points (no hardcoded imports)
- Added two entry point groups: ai_runner.pipeline and ai_runner.pipeline_params
- Removed Params class attribute from Pipeline interface (params discovered via entry points)
- Updated Dockerfiles to use pyproject.toml for dependency installation
- Removed requirements.live-ai.txt (deps consolidated into pyproject.toml)
- Updated all documentation with plugin system details
- Added Docker image hierarchy documentation to AGENTS.md

Benefits:
- Pipelines can live in separate repos and be installed via pip/uv
- No code changes needed in ai-runner when adding new pipelines
- Params can be parsed without importing heavy dependencies (torch, etc.)
- Works seamlessly with multiprocessing spawn context
- Cleaner separation of concerns between runtime and pipeline implementations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants